Methods, systems, and computer readable media for controlling presentation and selection of objects that are digital images depicting subjects

ABSTRACT

The subject matter described herein is directed to controlling presentation and selection of objects that are digital images depicting subjects. According to one aspect, a plurality of objects is presented via a presentation device having a screen. Each object is associated with at least one other object of plurality of objects and is a digital image of a subject. A first object presented is selected in response to a first selection event associated with the first object. A second object non-adjacent to the first object is determined based on a predefined relationship associating the first object with the second object, wherein the second object is on-screen or off-screen. The second is presented via the presentation device on-screen and adjacent to the first object in response to the determination. An indication of the predefined relationship is also presented.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/682,148 titled “Methods, Systems, and Computer Readable Media ForControlling Presentation and Selection of Objects That Are DigitalImages Depicting Subjects,” filed on Apr. 9, 2015 (now U.S. Pat. No.9,235,336 issued Jan. 12, 2016), which is a continuation of U.S. patentapplication Ser. No. 14/215,250 titled “Methods, Systems, and ComputerReadable Media For Controlling Presentation and Selection of ObjectsThat Are Digital Images Depicting Subjects,” filed on Mar. 17, 2014 (nowU.S. Pat. No. 9,015,635 issued Apr. 21, 2015), which is a continuationof U.S. patent application Ser. No. 13/246,215 titled “Methods, Systems,and Computer Readable Media For Controlling Presentation and Selectionof Objects That Are Digital Images Depicting Subjects,” filed on Sep.27, 2011 (now U.S. Pat. No. 8,694,921 issued Apr. 8, 2014), which is acontinuation of U.S. patent application Ser. No. 11/956,920 titled“Methods, Systems, and Computer Readable Media For ControllingPresentation and Selection of Objects That Are Digital Images DepictingSubjects,” filed on Dec. 14, 2007 (now U.S. Pat. No. 8,037,425 issuedOct. 11, 2011, the entire disclosures of which are hereby incorporatedby reference herein.

BACKGROUND

Digital images, such as photographs taken using digital cameras, scannedimages, and images generated with the aid of a computer, often containmultiple subjects. For example, a digital image may depict multiplepeople, places, or objects.

In order to facilitate later viewing or interpretation of digitalimages, it may be desirable to associate with the digital imagesinformation concerning the subjects depicted in the images. For example,if an image is a digital photograph, it may be desirable to associatewith the digital photograph information concerning the subjects depictedin the photograph. In addition, when using a digital camera to take aphotograph, it may be desirable to pre-associate with the photographinformation concerning subjects that will be depicted in the photograph.

Current digital photograph editing software allows manual annotation ofphotographs. However, manual annotation is labor intensive and istherefore undesirable when large numbers of photographs requireannotation. Current digital camera user interfaces allow the user toautomatically annotate digital photographs with time and dateinformation. However, there is no easy method to navigate and selectsubjects of a digital photograph either prior to or after creation ofthe photograph.

Accordingly, there exist a need for methods, systems, and computerreadable media for controlling presentation and selection of objectsthat are digital images depicting subjects.

SUMMARY

The subject matter described herein includes methods, systems, andcomputer readable media for controlling presentation and selection ofobjects that are digital images depicting subjects. According to oneaspect, the subject matter described herein includes a method forcontrolling presentation and selection of objects that are digitalimages depicting subjects. The method includes presenting, via apresentation device having a screen, a plurality of objects. Each of atleast one object is associated with at least one other object of theplurality of objects. Each of the objects is a digital image depicting asubject. A portion of the plurality of objects is off-screen. The methodfurther includes selecting a first object presented in response to afirst selection event associated with the first object. The methodfurther includes determining a second object based on a predefinedrelationship associating the first object with the second object. Thesecond object is at least one of on-screen and off-screen. The methodfurther includes presenting, via the presentation device, the secondobject on-screen and adjacent to the first object in response to thedetermination and an indication of the predefined relationship.

According to another aspect of the subject matter described herein, asystem for controlling presentation and selection of objects that aredigital images depicting subjects is provided. The system includes anobject selection rendering subsystem for presenting, via a presentationdevice having a screen, a plurality of objects. Each of at least oneobject is associated with at least one other object of the plurality ofobjects. Each of the objects is a digital image depicting a subject. Aportion of the plurality of images is off-screen. The system furtherincludes a navigation and selection subsystem for selecting a firstobject presented in response to a first selection event associated withthe first object. The system further includes an object dataset accesscomponent for determining, using an object dataset, a second objectbased on a predefined relationship associating the first object with thesecond object. The second object is at least one of on-screen andoff-screen. The object selection rendering subsystem is adapted topresent, via the presentation device, the second object on-screen andadjacent to the first object in the response to the determination and anindication of the predefined relationship.

According to another aspect of the subject matter described herein, asystem for controlling presentation and selection of objects that aredigital images depicting subjects is provided. The system includes meansfor presenting, via a presentation device having a screen, a pluralityof objects. Each of at least one object is associated with at least oneother object of the plurality of objects. Each object is a digital imagedepicting a subject. A portion of the plurality of objects isoff-screen. The system further includes means for selecting a firstobject presented in response to a first selection event associated withthe first object. The system further includes means for determining asecond object based on a predefined relationship associating the firstobject with the second object. The second object is at least one ofon-screen and off-screen. The system further includes means forpresenting, via the presentation device, the second object on-screen andadjacent to the first object in response to the determination and anindication of the predefined relationship.

According to another aspect, a computer readable medium embodying acomputer program executable by a machine, for controlling presentationand selection of objects that are digital images depicting subjects isprovided. The computer program includes executable instructions forpresenting, via a presentation device having a screen, a plurality ofobjects, wherein each of at least one object is associated with at leastone other object in the plurality of objects and a portion of theplurality of objects is off-screen. Each object is a digital imagedepicting a subject. The computer program further includes instructionsfor selecting a first object presented in response to a first selectionevent associated with the first object. The computer program furtherincludes instructions for determining a second object based on apredefined relationship associating the first object with the secondobject. The second object is at least one of on-screen and off-screen.The computer program further includes instructions for presenting, via apresentation device, the second object on-screen and adjacent to thefirst object in response to the determination and an indication of thepredefined relationship.

BRIEF DESCRIPTION OF THE DRAWINGS

Preferred embodiments of the subject matter described herein will now beexplained with reference to the accompanying drawings of which:

FIG. 1 is a flow chart illustrating an exemplary process for controllingpresentation and selection of objects that are digital images depictingsubjects according to an embodiment of the subject matter describedherein;

FIG. 2 is a block diagram illustrating a camera and exemplary softwarecomponents that may reside on the camera;

FIG. 3 is a block diagram illustrating an exemplary system forcontrolling presentation and selection of objects that are digitalimages depicting subjects according to an embodiment of the subjectmatter described herein;

FIG. 4 is a block diagram illustrating an exemplary object selection andpresentation control instance in which objects that are digital imagesdepicting subjects are arranged in vectors according to an embodiment ofthe subject matter described herein;

FIG. 5A is a block diagram illustrating an exemplary of hierarchy ofobjects that may be stored in an object dataset according to anembodiment of the subject matter described herein;

FIG. 5B is a block diagram illustrating an exemplary hierarchy ofobjects that may be stored in an object dataset according to anembodiment of the subject matter described herein;

FIGS. 6A-6C are block diagrams illustrating an object selection andpresentation control instance where objects are arranged in vectors andillustrating selection and presentation of objects in different vectorsaccording to an embodiment of the subject matter described herein; and

FIGS. 7A and 7B are tables illustrating exemplary relationships betweenobjects according to an embodiment of the subject matter describedherein.

DETAILED DESCRIPTION

The subject matter described herein includes methods, systems, andcomputer readable media for controlling presentation and selection ofobjects that are digital images depicting subjects. The followingdefinitions are used herein to describe certain aspects of the subjectmatter described herein.

-   -   Emphasis Selection Event: A specialized selection event on a        device that both selects an item and performs an action within        the context of an item. An example embodiment of this type of        event can be generated by a double click on a button of a        personal computer pointing device, such as a mouse.    -   Immutable Relationship: A subject that has an immutable        relationship to another subject in a relationship hierarchy has        a relationship that can never be altered. For example, an        offspring subject has a relationship to his or her parents that        is immutable. The parent-offspring biological relationship is        durable forever. Once the offspring is born, the relationship        cannot be altered or rescinded, even upon the death of the        subject or the offspring, the relationship remains true.    -   Coincidental Relationship: A subject has a coincidental        relationship to another subject in the hierarchy if the        relationship is non-immutable. A coincidental relationship may        be formed between two subjects based on a contract. For example,        a spouse of a subject is coincidentally related to the subject        through the contract of a marriage. An adopted child and        stepchild are also examples of children who may be        coincidentally related to a parent. Coincidental and immutable        relationships are not necessarily unique to family situations.

According to one aspect, the subject matter described herein includes amethod for controlling presentation and selection of objects that arerepresented by digital image subjects. FIG. 1 is a flow chartillustrating an exemplary method for controlling presentation andselection of objects that are digital images depicting subjectsaccording to an embodiment of the subject matter described herein.Referring to FIG. 1, in block 102, a plurality of objects arranged in aplurality of vectors is presented via a presentation device, whereineach object in a second vector is associated with an object in a firstvector and wherein each object is a digital image depicting a subject.For example, objects that are digital image thumbnails of subjectsdepictable in digital photographs, digital video frames,computer-generated images, or scanned images may be presented inconstructs referred to herein as vectors. Examples of presentation ofobjects in vectors will be described in detail below. Each object in onevector may be associated with an object in another vector. For example,one vector may contain objects representing one generation in a familyand another vector may contain objects representing a subsequentgeneration.

The subject matter described herein includes means for presenting, via apresentation device, a plurality of objects arranged in a plurality ofvectors, wherein each object in a second vector is associated with anobject in the first vector and wherein each object is digital imagedepicting a subject. For example, in FIG. 2, which illustrates a cameraand exemplary software components of a camera, software that controlspresentation and selection of digital images may be implemented inoperating system 202 of camera 204 or in an application that runs onoperating system 202. Other software that may reside on camera 204 is acommunication client 206 which communicates with another device andphoto files 208, which may be the digital images for which subjects areidentified. FIG. 3 illustrates exemplary components of an objectpresentation and control system 300 according to an embodiment of thesubject matter described herein. Referring to FIG. 3, an objectselection rendering subsystem 302 performs the operations listed inblock 102 of FIG. 1. For example, object selection rendering subsystem302 performs rendering for object selection and control. Objectselection rendering subsystem 302 renders information for each subjectby retrieving media objects and subject information, including imagethumbnails, from object dataset 304. In one implementation, the subjectinformation retrieved for each object may be associated as metadata witha digital image that depicts each subject. An object dataset accesscomponent 320 may facilitate extraction of objects and associatedinformation from object dataset 304. Object dataset 304 may beimplemented using a hierarchical data store that stores data for eachobject to be presented and/or selected.

In the context of usage within a digital camera, object presentation andselection control system 300 may be implemented as an application thatcreates an instance of object selection and presentation control. FIG. 4is a block diagram illustrating an instance of an object selection andpresentation control that may be created by system 300 according to anembodiment of the subject matter described herein. Referring to FIG. 4,object selection and presentation control instance 402 may include agraphical presentation of objects that are digital images depictingsubjects, where the objects are arranged in vectors. In the illustratedexample, each vector 404A-404D is a column of objects 406, 408, 410,412, 414, 416, 418, and 420 that are digital images depicting subjects.In the illustrated example, vector 404A represents parent objects,vector 404B represents sibling objects, vector 404C represents childrenobjects, and vector 404D represents grandchildren objects. In oneexemplary implementation, each vector 404A-404D may be a circularconstruct with a portion of the objects in each construct visible on adisplay, a portion of the objects in each construct not visible on thedisplay, and where visible and non-visible portions in each constructare linked together such that a last displayed object in the constructwill be followed by the display of a first displayed object in theconstruct in response to a user navigation event. Using vector 404A asan example, objects 406, 408, and 410 are visible to a user. Otherobjects may be off screen such that when user navigation event isdetected, for example, actuation of one of keys 422 and 424, off screenobjects in vector 404A become visible and currently displayed objects invector 404A may be scrolled off screen.

When object selection and presentation control instance 402 isinstantiated, object data application programming interface (API)component 316 illustrated in FIG. 3 may load object selection andpresentation control instance 402 with information related to eachobject. Calls to API 316 may be processed by object dataset accesscomponent 320, and object dataset 304 may be loaded with each subject'sinformation, including relationship information. System 300 may callobject data API component 316 passing an object, (e.g., an imagethumbnail of a subject), relationship information for other relatedobjects, and a correlation ID for the object. At a later time whenobject selections are retrieved by system 300, the correlation IDs maybe used by system 300 to identify selected objects. Other subjectinformation may be stored in object dataset 304. For example, as will bedescribed in detail below, media phrases and synthesized speech may bestored for each object to support speech and other media functionsassociated with object presentation and selection.

Object dataset 304 may include any suitable structure that allowsrelationships between objects to be defined and that allows storage orassociation with information identifying the subjects corresponding toeach object. In one example, dataset 304 may be implemented as ahierarchical dataset. In the hierarchical dataset, each record indataset 304 may be a node in a tree hierarchy. Each node may contain anumber of items, examples of which will now be described.

One data item that may be included in each node is an image thumbnail.The image thumbnail may be presented by object selection andpresentation control instance 402. Subjects may be selected by visualrecognition of the corresponding object displayed by object selectionand presentation control instance 402. Objects 406-420 are examples ofimage thumbnails that may be stored in object dataset 304. Another dataitem that may be stored in a record of object dataset 304 is a recordpointer or record pointer array, which can be used to reference othernode references in object dataset 304. It is these nodes and referencesto other nodes within a node that can be used to form a relationshiphierarchy.

Yet another data item that may be stored in a record in dataset 304 is anode type or node type array. This array can store an indicator for eachrecord pointer with the following values. If the value ofnode_type[current-subject-record-number] is “peer_node” subject then thereference node is peer of the subject in the current node. Subject peersare considered in the same generation as the subject. If the value ofnode_type[current-subject-record-number] is “child_node” then thesubject in the reference node is a logical child of the subject in thecurrent node. Child subjects are considered to be in the nextgeneration.

Yet another data item that may be stored in each record in objectdataset 304 is a relation type or relation type array. This array holdsan indicator for each record pointer with the following values: If thevalue of node_type[current-subject-record-number] is “immutable” thenthe subject in the reference node has an immutable relationship to thepeer or child being referenced. An example of an immutable relationshipis the relation between a father and a biological son. If the value ofnode_type[current-subject-record-number]is “coincidental” then thesubject in the referenced node has a coincidental relationship. Anexample of a coincidental relationship is the relationship between afather and a stepson. Each level of nodes in dataset 304 may represent ageneration.

In one exemplary implementation, subjects in the hierarchy in the samelogical generation are displayed in the same column vector in objectselection and presentation control instance 402. In the exampleillustrated in FIG. 4, vector 404A corresponds to a first display columnin object selection and presentation control instance 402 and maycontain objects in a first generation. Vector 404A may contain root nodeof the hierarchy of nodes stored in object dataset 304. Vector 404B maycontain a subordinate generation of objects. In object selection andpresentation control instance 402 illustrated in FIG. 4, eachsubordinate generation in the hierarchy is displayed to the right of theprevious generation. However, any suitable arrangement can be used toillustrate relationships between objects. For example, subordinategenerations could be stored to the left of, above, or below previousgenerations.

FIG. 5A is a block diagram illustrated in an example of a hierarchy thatmay be maintained in object dataset 304. In FIG. 5A, object 502 islocated at the root node of the hierarchy. Object 506 is a peer of rootobject 502. Dotted line 504 denotes object 506 as being a coincidentalpeer of root object 502. In the context of a family, the subjectrepresented by object 506 could be a spouse or a step-sibling of thesubject represented by object 502. Objects 502 and 506 are in the samelevel of the hierarchy and may be displayed in the same vector in objectselection and presentation control instance 402.

If an object is a child node, then that object is considered to be inthe next subordinate generation relative to the current generation. InFIG. 5A, objects 512, and 514 are children of object 502. Solid lines508 and 510 represent an immutable relationship between object 502 andobject 514. Object 518 has a coincidental relationship, such asmarriage, with object 514 as indicated by dotted line 516.

In object selection and presentation control instance 402, nodes 512,514, and 518 would be displayed in the next adjacent column to thecolumn where objects 502 and 506 are displayed. The relationship betweenthe nodes would be dictated by the hierarchy in object dataset 304. Inobject selection and presentation control instance 402 illustrated inFIG. 4, objects 412 and 414 in vector 404B may be the next subordinategeneration of objects in relation to objects 406, 408, and 410 stored invector 404A.

Objects 528 and 530 are children of object 512. Solid lines 520 and 522represent immutable relationships between object 512 and objects 528 and530. Similarly, objects 532 and 534 are children of object 514. Solidlines 524 and 526 represent immutable relationships between object 514and objects 532 and 534.

One data item that may be represented in object dataset 304 is arelationship state indicator that indicates whether the referenced nodeis an immutable relationship node or a coincidental relationship node.Object dataset 304 may determine whether the selected node is acoincidental relationship node based on the relationship state indicatorduring the processing of the selection event, as will be described indetail below.

The references stored in each record in dataset 304 determinerelationships between related nodes and their respective generations.The relationships may be used to control presentation of objects andaffect selection operations performed using object selection andpresentation control instance 402.

Object dataset access component 320 may return an image thumbnail foreach object in database 304 for display in the display area of objectselection and presentation control instance 402. In FIG. 4, objectselection and presentation control instance 402 appears on a displayscreen on a digital camera. Actuators 422, 424, 426, and 428 may allow auser to navigate between displayed objects. Actuator 430 may allow auser to select a displayed object. In the illustrated example, aselection cursor indicated by a bold dashed box appears around object408. When the user presses actuator 430 the first time that theselection cursor is on an object, that object may be selected.

Object selection and presentation control instance 402 allows a user toselect subjects for a photograph before or after the photograph istaken. In one implementation, a camera may be pre-loaded with images andrelationships of potential subjects.

Although in the example illustrated in FIG. 4, objects are arranged invertical vectors, any suitable arrangement for displaying objects may beused. For example, members of a given generation can be arranged in ahorizontal direction with additional generations displayed in adjacenthorizontal rows above or below the initial horizontal row. In addition,in a three dimensional space, an initial layout direction for the firstvector may be along any display vector in the three dimensional space,with the adjacent display vectors being arranged adjacent and parallelto the first display vector.

Returning to FIG. 1, in block 104, a method for selecting andcontrolling display of objects that are digital images depictingsubjects includes selecting a first object presented in a first vectorin response to a first selection event associated with the first object.For example, in FIG. 4, object 408 in vector 404A may be selected byplacing the cursor over object 408 using selection controls 422, 424,426, or 428 and actuating selection actuator 430. Accordingly, a systemfor controlling selection and presentation of objects that are digitalimages depicting subjects includes means for selecting first objectpresented in the first vector in response to a first selection eventassociated with the first object. For example, navigation and selectionsubsystem 317 illustrated in FIG. 3 may process navigation and selectionevents implemented using object selection and presentation controlinstance 402 illustrated in FIG. 4. Navigation and selection subsystem317 may include a navigation component 308 that processes navigationevents, simple and hierarchical selection components 310 and 312 thatprocess selection events, and voice and cursor pad input components 324and 306 that respectively process voice and tactile input for selectionand/or navigation. Navigation and selection subsystem 317 may outputselection events 326 to an associated application that has created andis using an instance of the control for the selection of subjects.

In FIG. 4, actuators 422, 424, 426, 428, and 430 may be implementedusing hardware buttons. In an alternate implementation, the actuatorsmay be displayed on an LCD panel and may be activated by touch. Otherkinds of input devices may be used to provide actuation, selection, andnavigation events. Exemplary input devices suitable for use withembodiments of the subject matter described herein include a personalcomputer based pointed device, such as a mouse or a joystick. On amobile phone, a cursor and hardware or software implemented actuators tocontrol selection of objects using a cursor may be used utilized. Inaddition, as will be described in more detail below, media phraserecognition may be used to select objects.

In FIG. 4, object selection and presentation control instance 402 maysupport at least the following modes of selection: simple selection modeand hierarchical selection mode. Simple selection mode will be describedin detail below. Hierarchical selection allows multiple objects to beselected in object selection and presentation control instance 402without having to navigate each object and select the object.Hierarchical selection may begin with selection of an initial object.The initial object can be located at any position in the display area ofobject selection and presentation control instance 402.

Hierarchical selection may begin when object selection and presentationcontrol instance 402 receives via cursor pad input component 306 anemphasis selection event. In the context of a digital camera as shown inFIG. 4, an emphasis selection event may be generated by pressingselection actuator component 430 twice in rapid succession.

An emphasis selection event may be similar to a double click event thatis generated using a pointing device of a personal computer. Somecameras may also support a press and hold function by selection actuatorcomponent 430 to generate the emphasis selection event. In this case,the button is pressed and held in the pressed state until a timethreshold is exceeded and the button is released. Some cameras mayprovide tactile or audible feedback when the hold time threshold isexceeded to notify the user that the emphasis selection function hasbeen invoked.

Upon receipt of input, cursor pad input component 306 forwards theemphasis selection event to hierarchical selection component 312 forprocessing. The selection event contains an x and y location of theselection cursor within object selection and presentation controlinstance 402 of FIG. 4. From the x and y location of the selectioncursor, simple selection component 310 queries object dataset 304 viaobject dataset access component 320 for the object that is displayed atthe cursor location. Hierarchical selection component 312 causes objectdataset access component 320 to set the state of the object as“selected.” Object dataset access component 320 causes object renderingsubsystem 302 to redraw the selected state of the object in objectselection and presentation control instance 402. The object then appearsas selected, as shown in by object 612 of FIG. 6B. In FIG. 6B, the linescrossing object 612 indicate selection.

In order to select an object using object selection and presentationcontrol instance 402, the selection cursor is positioned in the desiredobject cell. The selection cursor is initially positioned at a defaultcell location in object selection and presentation control instance 402.The default cell may be any cell within object selection andpresentation control instance 402. The initial position of the selectioncursor may be set programmatically before object selection andpresentation control instance 402 is displayed based on a default orspecified condition. An example condition may be the detection of acontext of a photographic occasion. For example, if it is determinedthat the event is uncle Joe's birthday, then the selection cursor may beinitially positioned on an object cell that contains a digital image ofuncle Joe.

The selection cursor is moved in response to navigational events fromnavigation actuators 422, 424, 426, and 428. Each navigation actuator422, 424, 426, and 428 may graphically indicate the direction ofmovement as part of the presentation of each actuator. For example,navigation actuator 424 points in the downward direction. When thisactuator is pressed, the cursor will be moved in the downward directionto the next object displayed by object selection and presentationcontrol instance 402. If the cursor is already at the bottom edge of thedisplay, then the vector in which the cursor is positioned will bescrolled if there are additional objects in the vector that are notcurrently displayed. As described above, each vector 404A-404D may be acircular construct, so that navigation events in the same direction willeventually cause the display of objects to wrap around and objects thatscrolled off screen will be displayed again.

An upward navigation event from a navigation actuator works in the samemanner at the top of the display as downward navigation event at thebottom of the display. When the cursor is at the top of a vector and anupward navigation event is received, the objects in the vector where thecursor is located will be scrolled in the downward direction.

Left or right navigation may be implemented differently than verticalnavigation. The cursor may be moved in left and right directions uponreceipt of left and right navigation event from left and rightnavigation actuators 426 and 428. If the cursor is already on the leftedge of the display and if additional vectors are logically available tothe left edge of the left edge vector, then the entire contents ofobject selection and presentation control instance 402 may be scrolledin a right direction, introducing a new column on the left. Using theexample of FIG. 4, when this type of scrolling is actuated, agrandparent column may be introduced on the left edge of the display,and the grandchildren column may be moved off the display on the right.

When a right navigation event if received, the selection cursor may bemoved to the right one cell in object selection and presentation controlinstance 402. If the cursor is already at the right edge of the displayand if additional vectors are logically available to the right of theright edge vector, then the entire contents of object selection andpresentation control instance 402 may be scrolled in a left direction,introducing a new column on the right. Using the example illustrated inFIG. 4, when this type of scrolling is actuated, a great-grandchildrencolumn will be introduced on the right edge, and the parent column willmove off the display on the left.

The supported cursor movements will allow the selection cursor to bepositioned on any object displayed in object selection and presentationcontrol instance 402. In addition, the movements allow off screenobjects to be moved on screen.

Returning to FIG. 1, in block 106, a method for controlling presentationand selection of objects that are digital images depicting subjectsincludes determining a second object, non-adjacent to the first object,based on a predefined relationship associating the first object with thesecond object. Referring to FIG. 6A, object 606 may be non-adjacent toobject to 602. The relationship between objects 606 and 602 may bestored in object dataset 304 illustrated in FIG. 3. A system forcontrolling presentation and selection of objects that are digitalimages representing subjects includes means for determining a secondobject, non-adjacent to the first object based on a predefinedrelationship associating the first object with the second object. Asdescribed above, dataset 304 may contain a hierarchy of objects beingdisplayed for selection in object selection and presentation controlinstance 402. Within dataset 304, the object hierarchy may be stored asa tree. In a tree organization, each node in the tree can maintainreferences to other nodes in the tree. In dataset 304, each referencemay contain additional data items that provide information aboutrelationships between objects.

One data item that may be included in each node is a state indicatorthat indicates whether the referenced node is a child node or a peernode. If a node is a child node, then that node is considered to be inthe next subordinate generation. In object selection and presentationcontrol instance 402, this node will be displayed in the next adjacentcolumn that is displayed parallel and adjacent to the parent column asdirected by object dataset 304. For example, objects in vector 404B isdisplayed to the right of objects in vector 404A in object selection andpresentation control instance 402, indicating that objects in vector404B are children of objects in vector 404A. Stated differently, objectsin vector 404B represent the next subordinate generation in relation toobjects in vector 404A. Peer nodes are displayed in the same vector inobject selection and presentation control instance 402.

Another data item that may be stored in a node reference in dataset 304is a relationship state indicator that indicates whether the referencednode is an immutable relationship node or coincidental relationshipnode. Object dataset 304 determines whether a selected node is acoincidental node based on the relationship state indicator during theprocessing of a selection event as described above.

The references stored with each node determine relationships betweenrelated nodes between generations. The relationships may be used to bothdisplay and affect selection operations using object selection andpresentation control instance 402.

Returning to FIG. 1, in block 108, a method for controlling presentationand selection of objects that are digital images depicting subjectsincludes presenting the second object in a second vector adjacent to thefirst vector in response to the determination. A system for controllingpresentation and selection of objects that are digital images depictingsubjects may include means for performing the operations recited inblock 108. Referring to FIG. 4, when objects are first displayed inobject selection and presentation control instance 402, the objects neednot be presented in any particular order. However, an initial displayorder can be established prior to displaying object selection andpresentation control instance 402. An advantage of providing an initialdisplay order would be the proper placement of a potential first objecton the screen. It is not until the first object is selected by emphasisselection that order and layout become important.

In block 106 illustrated in FIG. 1, an object comprising a digital imagerepresenting a subject was selected using emphasis selection. When theobject is selected using emphasis selection, a selection and asubsequent action may occur. In object selection and presentationcontrol instance 402, the subsequent action contains the followingsteps: a sorting and positioning step and a subsequent additiveselection step.

Upon receipt of input, cursor pad input component 306 forwards theemphasis selection event to hierarchical selection component 312 forprocessing. The selection event contains an x and y location of theselection cursor within object selection and presentation controlinstance 402 of FIG. 4. From this x and y location of the selectioncursor, simple selection component 310 queries object dataset 304 viaobject dataset access component 320 for the object that is displayed atthe cursor's location. Hierarchical selection component 312 calls objectdataset access component 320 to set the state of the object as selected.Object dataset access component 320 then calls object sort component314. Object sort component 314 examines the currently selected node ornodes in the relationship hierarchy and assigns the nodes related to thecurrently selected hierarchy of nodes to be displayed in positions inobject selection and presentation control instance 402 that are adjacentto the first selected object. Object dataset access component 320 callsobject rendering subsystem 302 to redraw the selected state of theobject cell in object selection and presentation control instance 402.In the example illustrated in FIG. 6B, object 606, which was notadjacent to object 602 in FIG. 6A is moved to be adjacent to object 602.

FIGS. 6A and 6B demonstrate an exemplary sort process between theleftmost column and the next adjacent column to the right. In FIG. 6A,the selection cursor is positioned on object 602. Referring to FIG. 4,selection actuator component 430 may be actuated in a way that generatesan emphasis selection event. This event is processed and the selectionis determined as described above. The display result as shown in FIG. 6Bwhere object 612 is selected. In FIG. 6A, object 606 is not adjacent toobject 602. The sort process is applied and the sort determines thatobject 606 is related to object 602 while object 604 is not related. Thesource process updates the display positions of object 604 and 606. FIG.6B shows that object 606 is adjacent to object 602 (shown as 612 in FIG.6B). The sort and repositioning process continues across all of thegenerations in the relationship hierarchy that are represented inadjacent columns in object selection and presentation control instance402, whether the objects are displayed or hidden.

Once sorting has been completed, object dataset 304 calls objectrendering subsystem 302 to redraw the modified portions of the displayarea of object selection and presentation control instance 402,including objects in new positions, selection emphasis where cells areselected, and the selection cursor.

When selection actuator 430 is subsequently invoked for emphasisselection, the selection sort and display processes are again invoked.FIG. 6C shows the result of an additional cycle of the selectionprocess. After the additional cycle, object 606 is selected.

Two embodiments with respect to the object selection process will now bedescribed. Common to both embodiments of the selection process is theestablishment of an initial selection object. This object is selected atthe beginning of the selection process and defines the root node and therelationship hierarchy. From this root node, subsequent nodes areselected by traversing the relationship hierarchy and selecting nodes ineach succeeding level of the relationship hierarchy.

Selection with respect to coincidental relationship peer objects andcoincidental relationship children objects will now be discussed withregard to first and second embodiments. The first embodiment allowsautomatic selection of coincidental relationship peers and childrenobjects in the relationship hierarchy when immutable peers and childrenobjects in the relationship hierarchy are selected.

When a hierarchical selection begins, and progresses, the selectionprocesses that occur will be described with regard to FIG. 5A. FIG. 5Adepicts selection schematically when automatic selection of thecoincidental relationship peers and children subjects is in use. Eachselection rectangle indicates that the result after an emphasisselection cycle.

In FIG. 5A, object 502 has a coincidental peer relationship to object506. This coincidental peer relationship is shown by dotted connectingline 504. Selection rectangle 536 shows that object 506 is selected whenan emphasis selection is invoked on object 502. That is, object 502 isselected, and the coincidental relationship peer object 506 is alsoautomatically selected.

When emphasis selection is invoked again on object 502, the result isshown in selection rectangle 538, and objects 512, 514, and 518 areadded to the selection. Object 518 is also a coincidental relationshipobject with respect to object 514 as indicated by dotted line 516.Accordingly, multiple user interactions associated with the same objectmay automatically select related objects. For example, when emphasisselection is invoked again on object 502, the result is shown inselection rectangle 540, and objects 528, 530, 532, and 534 are added tothe selection.

The second selection embodiment does not perform automatic selection ofcoincidental relationship peers and children objects in the relationshiphierarchy when immutable peers and children objects in the relationshiphierarchy are selected.

FIG. 5B depicts schematically how selection operates when automaticselection of coincidental relationship peers and children objects is notperformed. Each selection rectangle indicates the result after anemphasis selection cycle.

In FIG. 5B, object 502 has a coincidental peer relationship with object506. This peer relationship is shown by the dotted connecting line 504.Selection rectangle 542 shows the objects selected when an emphasisselection is invoked on object 502. In the illustrated example,rectangle 542 indicates that when object 502 is selected, coincidentalrelationship peer object 506 is not also selected.

When emphasis selection is invoked again on object 502, the result isshown in selection rectangle 544. Coincidental relationship peer object506 has now been selected.

When emphasis selection is invoked again on object 502, the result isshown in selection rectangle 546. The children relationship objects 512and 514 have been selected, while coincidental relationship peer object518 to child node 514 has not been selected.

When emphasis selection is invoked again on object 502, the result isshown in selection rectangle 548. Coincidental relationship peer object518 to child node 514 is added to the selection.

When emphasis selection is invoked again on object 502, the result isshown in selection rectangle 550 and objects 528, 530, 532, and 534 areadded to the selection.

As shown by FIGS. 5A and 5B, allowing automatic selection ofcoincidental relationship objects saves steps in the selection processover a given set of objects. In some cases however, more granularselection may de desired. In those cases, automatic selection can beturned off or either inhibited completely through settings of objectselection and presentation control instance 402.

Because the hierarchical selection process is an additive process, oncean object has been selected, it remains selected until the entireselection is cleared. Clearing of a selection may be invoked by a clearall function implemented by object selection and presentation controlinstance 402. Individual objects may be deselected by moving theselection cursor to a selected cell and invoking a simple selectionusing selection actuator 430. The cell then becomes deselected, but theselection of other selected cells remains unaffected.

When simple selection is used, a single object cell, at the selectioncursor, upon receipt of the selection event, is selected. If the cell iscurrently already selected, the selection event will deselect the cell.Use of simple selection is optimal when small numbers of objects are tobe selected or a larger selection of objects needs to be pruned ortrimmed by a small number of objects.

Simple selection occurs when a simple selection event is captured bycursor pad input component 306 of FIG. 3. Cursor pad input component 306passes the event to simple selection component 310 for processing. Theselection event contains and x and y location of the selection cursorwithin object selection and presentation control instance 402 of FIG. 4.From this x and y location of the selection cursor, simple selectioncomponent 310 queries object dataset access component 320 for the objectthat is displayed at the cursor location. Simple selection component 310then determines if the object and the cursor had been previouslyselected. In FIG. 6B, object 612 is enclosed by the selection cursor andis selected. Object 602 of FIG. 6A shows the same object in the cell butthe cell is not selected.

If an object is in the selected state when a simple selection event isreceived, the object is deselected by simple selection component 310. Inorder to deselect an object, simple selection component 310 calls objectdatabase access component 320 to update the selection state of theobject in object dataset 304 to “deselected.” Object dataset accesscomponent 320 calls object rendering subsystem 302 to redraw theselection state of the object in object selection and presentationcontrol instance 402.

If an object is not in the selected state when a simple selection eventis received, the object is selected by simple selection component 310.In order to select the object, simple selection component 310 callsobject dataset access component 304 to update the selected state of theobject. Object dataset access component 320 calls object renderingsubsystem 302 to redraw the selection state of the object cell in objectselection and presentation control instance 402. When simple selectionis used, subsequent sorting and positioning in adjacent columns may notbe performed.

In the examples described above, selection of objects occurs throughtactile events. In an alternate implementation, selection may occurthrough media recognition, such as voice recognition. For example,object dataset 304 may be preloaded with a media phrase representingeach object. If the object is a digital image of a person, the mediaphrase may be recorded speech of the person or a media phraseidentifying the person by name. In one example, voice prints may berecorded. A voice print is an encoded utterance of speech that can becompared to an utterance from a human subject to determine or identifythe human subject. When voice identification of subjects is used,dataset 304 may be loaded with voice prints for each object. The voiceprints for each object may be added to object dataset 304 along with theimage thumbnail and any metadata associated with each object.

When a microphone on the camera (not shown) captures an utterance, theutterance is recorded in memory. A voice event containing a reference tothe utterance is presented to object selection and presentation controlinstance 402 via voice input component 324. Voice input component 324compares the utterance to voice prints in object dataset 304 anddetermines that the utterance matches a voice print for an object inobject dataset 304. If a match is found in dataset 304, voice inputcomponent 324 passes a reference to the object to simple selectioncomponent 310 with an indication to mark the object as selected. Voicerecognition of an object may result in selection of the object. In orderto avoid oscillation between selection and de-selection, an object maynot be deselected as a result of voice recognition.

Thus, when voice or other media phrase recognition is used, a camera maybe activated prior to taking a photograph and voice recognition may beused to pre-select objects corresponding to subjects that may bedepicted in the photograph.

In another enhancement of the subject matter described herein, a cameramay include a speech synthesis system than when called with text data,converts the data to speech output. Using this output mechanism, objectselection and presentation control instance 402 can be enhanced so thatwhen control instance 402 is rendered visually on the camera's display,the speech synthesis system can also audibly enumerate a list ofdisplayed or select objects.

The subject matter described herein can be further enhanced by adding amechanism as disclosed in a commonly-assigned, co-pending patentapplication entitled Methods, Systems, and Computer Readable Media forAssociating Information with a Digital Image, the disclosure of which isincorporated herein by reference in its entirety. The commonly assigned,co-pending patent application discloses, in one example, a method forautomatically identifying family members in a photograph using metadatain the photograph and a genealogy database. Thus, if objects displayedin object selection and presentation control instance 402 are people,dataset 304 may store genealogical relationships between the people tofacilitate selection of subjects to be included in a digital image. Ifthe objects being displayed are goods or services, such as automobilemakes and models, dataset 304 may store relationships between automobilemakes and models to facilitate identification of image subjects.Similarly, if the objects displayed in a photograph are places, dataset304 may store relationships between places to facilitate identificationof places in the photograph.

Relationship information stored in dataset 304 can be used to customizethe enumeration of subjects to be contextually sensitive to thephotographer. For example, the relationships displayed may be displayedfrom the viewpoint of the photographer based on the photographer'srelationship to each object in dataset 304.

Continuing with the speech synthesis example, during the object controldisplay rendering process, object rendering subsystem 302 of FIG. 3 maycall an internal object speech rendering component (not shown) toenumerate in an audio stream a list of selected objects. Object speechrendering component 322, starting at the root node selected in thedatabase, may search for an object that matches the name of thephotographer.

If the photographer is not found to be an object in object dataset 304,then the photographer has no known relationship to the objects in thedatabase. In this case, the photographer's relationship is assumed to beunrelated to the objects, and the speech output stream is generatedusing the names of the objects, but no relationship information isincluded in the speech output stream. If the photographer is an objectin dataset 304, object speech rendering component 322, using dataset304, determines the relationships of each object to the photographer anduses this information in each subject's name in building the speechoutput stream. In this case, the speech stream is personalized to thephotographer, allowing the photographer to understand not only eachobject, but the object's relationship to the root node. An example willdemonstrate this enhancement. In the example, dataset 304 may containthe following information:

-   -   Father: Joe Smith    -   Son: David Smith (Photographer)    -   Son: Roger Smith (Brother of David Smith, Son of Joe Smith)    -   Son: Christopher Smith (Son of Roger Smith)        David Smith is designated as the photographer (previously        specified, not shown). Object selection and presentation control        instance 402 is rendered using the relationship-customized        speech output stream and the following result is heard        (relationship data shown in italics):    -   “The selected subjects include father, Joe Smith, brother, Roger        Smith, and nephew, Christopher Smith.”        Without the personalization, the result would be:    -   “The selected subjects include Joe Smith, Roger Smith, and        Christopher Smith.”        Thus, perspective information may be used to customize the        presentation of objects in object selection and presentation        control instance 402.

As described above, one potential use for the subject matter describedherein is automatically associating metadata with a digital image. Forexample, metadata associated with objects selected in object selectionand presentation control instance 402 may be extracted from objectdataset 304. The extracted metadata may be automatically associated withan image in which the subjects represented by the first and secondobjects are depicted.

The subject matter described herein for controlling presentation andselection of objects representing digital image subjects may beimplemented on any suitable device capable of storing, creating, ordisplaying a digital image in which it is desirable to characterizesubjects. Examples of devices with which the present subject matter maybe utilized include cameras, camera phones, mobile phones, smart phones,and computers.

Although in the examples described above, the subjects are described asbeing people, any suitable subject matter displayable on a digital imagemay comprise a subject. Additional examples of subjects with which thepresent subject matter may be utilized include goods and geographiclocations.

As described above, in one embodiment, object selection and presentationcontrol instance 402 illustrated in FIG. 4 includes circular constructs.For example, each vector 404A-404D may comprise a circular constructwhere a portion of the objects in each construct is visible on thedisplay on a portion of the object in each constructs are not visible onthe display. The visible and non-visible portions in each construct maybe linked to each other such that a last displayed object in a constructwill be followed by display of a first displayed object in a constructin response to a user navigation event. For example, when the bottomobject in vector 404A is displayed, and the user selects control 424,the top most object in vector 404A may be displayed.

As stated above, relationships between objects may be coincidental orimmutable. FIGS. 7A and 7B are tables illustrating exemplarycoincidental and immutable relationships between entities. In FIG. 7A,examples of coincidental and immutable genealogical relationshipsbetween people are illustrated. In FIG. 7B, examples of coincidental andimmutable relationships between car makes and models are illustrated.The information depicted in FIGS. 7A and 7B may be stored in dataset 304illustrated in FIG. 3.

It should be understood that the various components illustrated in thevarious block diagrams represent logical components that are configuredto perform the functionality described herein and may be implemented insoftware, hardware, or a combination of the two. Moreover, some or allof these logical components may be combined, some may be omittedaltogether, and additional components can be added while still achievingthe functionality described herein. Thus, the subject matter describedherein can be embodied in many different variations, and all suchvariations are contemplated to be within the scope of what is claimed.

To facilitate an understanding of the subject matter described above,many aspects are described in terms of sequences of actions that can beperformed by elements of a computer system. For example, it will berecognized that the various actions can be performed by specializedcircuits or circuitry (e.g., discrete logic gates interconnected toperform a specialized function), by program instructions being executedby one or more processors, or by a combination of both.

Moreover, executable instructions of a computer program for carrying outthe methods described herein can be embodied in any machine or computerreadable medium for use by or in connection with an instructionexecution machine, system, apparatus, or device, such as acomputer-based or processor-containing machine, system, apparatus, ordevice, that can read or fetch the instructions from the machine orcomputer readable medium and execute the instructions.

As used here, a “computer readable medium” can be any means that cancontain or store the computer program for use by or in connection withthe instruction execution machine, system, apparatus, or device. Thecomputer readable medium can be, for example, but not limited to, anelectronic, magnetic, optical, electromagnetic, infrared, orsemiconductor machine, system, apparatus, or device. More specificexamples (a non-exhaustive list) of the computer readable medium caninclude the following: a portable computer diskette, a random accessmemory (RAM), a read only memory (ROM), an erasable programmable readonly memory (EPROM or Flash memory), an optical fiber, a portablecompact disc (CD), a portable digital video disc (DVD), and the like.

Thus, the subject matter described herein can be embodied in manydifferent forms, and all such forms are contemplated to be within thescope of what is claimed. It will be understood that various details ofthe invention may be changed without departing from the scope of theclaimed subject matter. Furthermore, the foregoing description is forthe purpose of illustration only, and not for the purpose of limitation,as the scope of protection sought is defined by the claims as set forthhereinafter together with any equivalents thereof entitled to.

The invention claimed is:
 1. A method for controlling presentation andselection of objects that are digital images depicting subjects, themethod comprising: presenting, via a presentation device having ascreen, a plurality of objects, wherein each of at least one object isassociated with at least one other object of the plurality of objectsand is a digital image depicting a subject and a portion of theplurality of objects is off-screen; selecting a first object presentedin response to a first selection event associated with the first object;determining a second object based on a predefined relationshipassociating the first object with the second object, wherein the secondobject is at least one of on-screen and off-screen; and presenting, viathe presentation device, the second object on-screen and adjacent to thefirst object and an indication of the predefined relationship, whereinat least one preceding action of presenting the plurality of objects,selecting the first object, determining a second object, and presentingthe second object is performed on at least one electronic hardwarecomponent.
 2. The method of claim 1 further comprising determiningmetadata associated with the first and second objects and automaticallyassociating the metadata with a digital image depicting subjectsdepicted in the digital images of first and second objects.
 3. Themethod of claim 1 further comprising selecting the second object inresponse to a second selection event associated with the first objectbased on the predefined relationship.
 4. The method of claim 3 whereinthe first and second selection events are user interactions with thefirst object.
 5. The method of claim 1 wherein the first selection eventcomprises one of a tactile selection event and a voice selection event.6. The method of claim 1 further comprising storing a media phrase foreach of the objects and wherein the first selection event comprisesautomatic recognition of a detected media phrase matching a media phrasestored for the first object.
 7. The method of claim 1 further comprisingstoring synthesized speech identifying the subjects with which each ofthe objects is associated and wherein presenting the plurality ofobjects includes playing the synthesized speech for at least one of theobjects.
 8. The method of claim 1 wherein the subjects comprise at leastone of people, goods, and geographic locations.
 9. The method of claim 8wherein the subjects comprise people and wherein the predefinedrelationship between the first and second objects comprises agenealogical relationship.
 10. The method of claim 1 wherein presentingthe plurality of objects in a plurality of rows and columns includespresenting the objects in circular constructs wherein a portion of theobjects in each construct is visible on the display device, a portion ofthe objects in each construct is not visible on the display device, andwherein the visible and non-visible portions in each construct arelinked to each other such that a last displayed object in a constructwill be followed by display of a first object in the construct inresponse to a user navigation event.
 11. A system for controllingpresentation and selection of objects that are digital images depictingsubjects, the system comprising system components including: an objectselection rendering subsystem for presenting, via a presentation devicehaving a screen, a plurality of objects, wherein each of at least oneobject is associated with at least one other object in the plurality ofobjects and is a digital image depicting a subject and a portion of theplurality of objects is off-screen; a navigation and selection subsystemfor selecting a first object presented in response to a first selectionevent associated with the first object; an object dataset accesscomponent for determining a second object based on a predefinedrelationship associating the first object with the second object,wherein the second object is at least one of on-screen and off-screen;and wherein the object selection rendering subsystem is adapted topresent, via the presentation device, the second object on-screen andadjacent to the first object and an indication of the predefinedrelationship, and wherein at least one of the system components includesat least one electronic hardware component.
 12. The system of claim 11wherein the object database access component is adapted to determinemetadata associated with the first and second objects and toautomatically associate the metadata with a digital image depictingsubjects depicted in the digital images of the first and second objects.13. The system of claim 11 wherein the navigation and selectionsubsystem is adapted to select the second object presented in responseto a second selection event associated with the first object based onthe predefined relationship.
 14. The system of claim 11 wherein thefirst selection event comprises one of a tactile selection event and avoice selection event.
 15. The system of claim 11 further comprising anobject dataset for storing the objects and media phrases for theobjects, wherein the first selection event comprises automaticrecognition of a detected media phrase matching a media phrase storedfor the first object.
 16. The system of claim 11 further comprising anobject dataset for storing the objects and synthesized speechidentifying the subjects with which the objects are associated andwherein the object selection and rendering subsystem is adapted to playthe synthesized speech for at least one of the objects.
 17. The systemof claim 11 wherein the subjects comprise at least one of people, goods,and geographic locations.
 18. The system of claim 17 wherein thesubjects comprise people and wherein the predefined relationship betweenthe first and second objects comprises a genealogical relationship. 19.The system of claim 11 wherein the object selection rendering subsystemis adapted to present the objects in circular constructs wherein aportion of the objects in each construct is visible on the displaydevice, a portion of the objects in each construct is not visible on thedisplay device, and wherein the visible and non-visible portions in eachconstruct are linked to each other such that a last displayed object ina construct will be followed by display of a first construct in theconstruct in response to a user navigation event.
 20. A system forcontrolling presentation and selection of objects that are digitalimages depicting subjects, the system comprising components including:means for presenting, via a presentation device having a screen, aplurality of objects, wherein each of at least one object is associatedwith at least one other object in the plurality of objects and is adigital image depicting a subject and a portion of the plurality ofobjects is off-screen; means for selecting a first object presented inresponse to a first selection event associated with the first object;means for determining a second object based on a predefined relationshipassociating the first object with the second object, wherein the secondobject is at least one of on-screen and off-screen; and means forpresenting, via the presentation device, the second object on-screen andadjacent to the first object and an indication of the predefinedrelationship, wherein at least one of the means includes at least oneelectronic hardware component.
 21. A non-transitory computer-readablemedium embodying a computer program executable by a machine, forcontrolling presentation and selection of objects that are digitalimages depicting subjects, the computer program comprising executableinstructions for: presenting, via a presentation device having a screen,a plurality of objects, wherein each of at least one object isassociated with at least one other object in the plurality of objectsand is a digital image depicting a subject and a portion of theplurality of objects is off-screen; selecting a first object presentedin response to a first selection event associated with the first object;determining a second object based on a predefined relationshipassociating the first object with the second object, wherein the secondobject is at least one of on-screen and off-screen; and presenting, viathe presentation device, the second object adjacent to the first objectin response to the determination and an indication of the predefinedrelationship.
 22. The computer readable medium of claim 21 wherein thecomputer program includes executable instructions for determiningmetadata associated with the first and second objects and automaticallyassociating the metadata with a digital image depicting subjectsdepicted in the digital images of the first and second objects.